Three dimensional (3D) models have many uses in modern communication, such as novel view generation. “Novel view generation” is using the video from several cameras to construct a view from a point where there is no physical camera. This is valuable to provide a fly-through view of a factory or a sports game or a meeting, for example. Reconstruction algorithms for 3D models suffer from problems, such as incompletion due to occlusions and inaccuracy. Using temporal information has proven useful for improving reconstruction quality in some cases with rigid structures or restricted motion.